Open Journal of Modern Linguistics, 2017, 7, 197-215

http://www.scirp.org/journal/ojml

ISSN Online: 2164-2834

ISSN Print: 2164-2818


Comparison between Peer Feedback and Automated Feedback in College English Writing: A Case Study


Yang Luo, Yan Liu

Faculty of Foreign Languages, Southwest Forestry University, Kunming, China




How to cite this paper: Luo, Y., & Liu, Y. (2017). Comparison between Peer Feedback and Automated Feedback in College English Writing: A Case Study. Open Journal of Modern Linguistics, 7, 197-215. https://doi.org/10.4236/ojml.2017.74015


Received: July 21, 2017

Accepted: August 26, 2017

Published: August 29, 2017


Copyright © 2017 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY 4.0). http://creativecommons.org/licenses/by/4.0/

Open Access

Abstract

As written feedback is an indispensable component of instructing and learn- ing process, the implementation of effective feedback plays a key role in im- proving non-English majors’ writing skill. Peers and automated writing eval- uation systems are new, main sources of feedback providers in college English writing. This paper compares three feedback conditions: individual and group modes in peer feedback and automated feedback. Analysis is made on distri- bution features from feedback types, dimensions of assessment rubric, aspects of organization and linguistic performance, and reflected attitudes. The find- ings indicate automated mode and group mode take turns dominating in non-corrective feedback and corrective feedback; group mode takes the lead in both direct and indirect feedback; individual, group and automated modes underemphasize global feedback. Dimensions are found to centre on linguis- tic performance, followed by organization, content and format, differences and similarities of specific aspects and attitudes are also found in all three modes. Further investigations are undertaken into students’ perceptions to- wards peer feedback and automated feedback, with their respective merits and demerits summed up. On the basis of all the findings, key elements of soci- ocultural theory are explored to provide multi-dimensional feedback scaf- folding for students of lower-intermediate level in facilitating college English writing.

Keywords

Peer Feedback, Automated Feedback, Distribution Features, Sociocultural Theory, Scaffolding, College English Writing



  1. Introduction

    L2 writing is an essential skill for non-English majors in colleges and universi-


    DOI: 10.4236/ojml.2017.74015 Aug.29, 2017 197 Open Journal of Modern Linguistics


    ties. Writing in English enables generating ideas, disseminating knowledge within specific disciplines and achieving effective communication in an interna- tional circle, which can promote non-English majors’ prospects in academic areas and job markets (Leggette et al., 2013; Raoofi et al., 2017). There is an in- creasing consensus that written feedback is an indispensable component of in- structing and learning process (Cho & Schunn, 2010; Leggette & McKim, 2013; Morch et al., 2017) and the implementation of effective feedback plays a key role in improving non-English majors’ writing skill (Forrer et al., 2015; Wei, 2016). Feedback provided by teachers, peers and automated writing evaluation systems makes the main sources of assessing English writing for college students (Lu, 2016; Wei, 2016; Aryadoust & Riazi, 2017). In Chinese college English writing, teacher feedback dominates while peer feedback and automated feedback are less frequently used (Bai, 2012; Zhou, 2013), which compromises the efficacy of feed- back in facilitating students’ writing competence. Recently, with student-centered teaching philosophy gradually accepted, peer feedback has been paid increasing attention to and practiced in language classes for its great significance in moti- vating students’ participation, cultivating their critical thinking and developing their ability of self-regulated learning (Plank & Condliffe, 2011; Bai, 2012; Liao & Yang, 2012; Forrer et al., 2015; Wang, 2016). Studies on peer feedback have re- vealed students can also be an information-and-assistance provider in L2 writing (Leggette et al., 2013; Wang, 2016), for they have sufficient time, energy and re- source (Andrade, 2008; Cho & Schunn, 2010; Luo, 2016). It is especially applica- ble to large-size college English classes in China. Comparatively, automated feedback is a new, heated subfield resulting from the development of technology and corpora. The existing relevant studies agree upon its undeniable value and emerging impact on L2 writing (Chapelle et al., 2015; Lu, 2016; Liu, 2016; Luo, 2016; Morch et al, 2017). However, every single feedback mode has its own ad- vantages and limitations, blind acceptance and adoption can only lead up to teaching passivity and deviate from teaching reality, weakening their potential effectiveness. For this reason, comparative studies can conduce to effective feed- back framework for college English writing with careful considerations into contributions of each mode to students’ writing competence. Previous compara- tive studies have been mainly focused on teacher feedback and peer feedback and drawn conclusions that peer feedback should be incorporated into teacher feedback, fulfilling the optimal efficacy of feedback (Tsui & Ng 2000; Yang, 2006; Xu & Liu, 2010; Bai, 2012; Yu, 2013; Forrer et al., 2015). However, few empirical studies have been made on comparison between peer feedback and automated feedback (Morch et al., 2017) or students’ perceptions towards them (Wei, 2015). Since strengths cannot offset weaknesses co-existing in each feedback mode, in what manner can combinations address this challenge? Answers can be sought from empirical comparative studies within a certain theoretical frame- work, for the advantages can be fully exploited and disadvantages comple- mented. Therefore, to explore an effective feedback package to better practice principles of student-centered learning, this paper compares peer feedback and


    automated feedback by analyzing their respective distribution features. Further investigations into students’ perceptions towards them are made and merits and demerits within summarized. On the basis of the findings, key elements of soci- ocultural theory are explored to provide feedback scaffolding for students of lower-intermediate level in facilitating college English writing.

  2. Literature Review

    1. The Sociocultural Thoery

      The sociocultural theory holds human mental functioning is essentially a social and cultural phenomenon and cognitive development is the result of consistent interactions between individuals and the surrounding historical contexts (Wertsch, 1985) by means of indirect connections playing the role of mediation (Vygotsky, 1978; Engeström, 1987). Mediation can be realized by tools and signs: with tools people can learn to control their behavior from the outside and with signs they can regulate their mind from the inside. Through exchanges be- tween the outside and the inside, internalization of meaning can be achieved (Vygotsky, 1978).

      The concept of mediation was expanded into activity theory (Engeström, 2001), emphasizing that human cognitive development was goal-directed and the result of individual activities in social interactions. Currently, triangle model from activity theory is the most widely adopted in writing research. Within this model, subject, object, mediation, rules, community and division of labor are key ingredients (Lantolf & Thorne, 2006: 222-224) and form sociocultural contexts (Lu, 2016). Subject cannot achieve the goal without means of mediation, whether in an individual or a collective mode. Object, written essays, is the achievement of the goal through efforts of subject from mediation and interaction. Rules in- clude assessment rubric and process mechanism for assessment. Teachers and students form the learning community and division of labor indicates the role and responsibility of the learning community during feedback. In social and cultural contexts subject can fulfill the goal with impacts and restrictions exerted by rules, community, mediation and division of labor. These key elements from sociocultural theory can provide the feedback scaffolding to develop EFL learn- ers’ English writing competence.

    2. Definition and Classification of Feedback

      Feedback in written forms refers to the input from readers to writers, providing information to modify essays (Keh, 1990; Zhu, 2010), thus, an interactive ex- change established (Rollinson, 2005: 27; Yang & Wu, 2011). In teaching research feedback is universally acknowledged as an important vehicle for improving and reinforcing learning (Zhou, 2013; Forrer et al., 2015). In this sense, effective feedback plays a crucial role in the development of EFL learners’ writing skills (Ferris, 2003: 19; Sarie, 2017).

      Written feedback can be classified into corrective feedback and non-corrective


      feedback. Corrective feedback promotes the acquisition of the second language and avoids its fossilization and regression by providing negative evidence for EFL learners (Wei & Shi, 2016) and improves their L2 writing and self-regulated learning (Wang & Liu, 2012). Non-corrective feedback scaffolds English writing in aspects of content, organization, linguistic performance and format, guiding learners in theme, cohesion means, structure, wording, phrasing, semantic clari- fication, grammar and syntax (Coyle & Roca de Larios, 2014).

      In terms of the way corrective feedback is presented, there are direct feedback and indirect feedback (Li & Ye, 2016). Direct feedback is a strategy which pro- vides direct comment or assistance to correct errors or improve the deficiencies while indirect feedback a strategy which marks the errors or deficiencies without giving direct corrections. The former is the easiest and quickest way to help writers accomplish higher-quality products (Chandler, 2003) whereas the latter assists them deepen understanding, motivate self-corrections and gradually sharpen writing skills (Ellis et al., 2008). With respect to the information correc- tive feedback contains, there are global feedback and local feedback (Chang, 2015). Global feedback refers to corrections focused on content, cohesion, or- ganization, format of essays while local feedback to corrections concerning grammar, syntax, vocabulary, collocation, punctuation, spelling, etc. Feedback of different types is beneficial for polishing up English essays.

      Currently, the intense interest is drawn in corrective feedback for its obvious and direct impact upon English writing improvement. An army of relevant stu- dies are mainly focused on direct-indirect feedback (Wang, Wei, 2015; Li & Ye, 2016; Wei & Shi, 2016; Sarie, 2017) or global-local feedback (Yang & Wu, 2011; O’Mahony et al, 2013; Wang, 2016). In a bid to capture all existing types in writ- ten feedback, there is a necessity to incorporate both non-corrective and correc- tive feedback, including different types of the latter into research.

    3. Research of Feedback

      Teacher feedback is a common practice to help students improve their writing skills in English writing class (Black & William, 2009; Wei, 2015), even though its effectiveness still remain debatable (Wei, 2015; Aryadoust & Riazi, 2017). Some scholars argue against teacher feedback in the failure to offer speedy feed- back (Bai, 2012), the negative result upon students to over-rely on their teachers and weaken their initiatives (Zhang, 2016). However, numerous empirical stu- dies have indicated that teacher feedback helps students learn more and better (Chandler, 2003; Morch et al. 2017), modify their essays towards greater gram- matical accuracy, pragmatic appropriateness, complexity, clarity and compre- hensibility (Li & Ye, 2016; Sarie, 2017), and thereby improve their writing com- petence (Wang & Liu, 2012; O’Mahony et al., 2013).

      Peer feedback is the comment provided by students of equal status in pairs or groups, with better results to follow the criteria (Bai, 2012; Leggette et al., 2013; Forrer et al., 2015). It is typical of student-centered L2 teaching and learning set- tings and often used in higher education (Wang, 2016; Ion et al., 2016). Empiri-


      cal studies on peer feedback find its effectiveness works in different aspects of writing: improvement of structure and content (O’Mahony et al., 2013; Wei, 2015), internalization of grammatical knowledge (Zhang, 2016) or better per- formance on content and linguistic performance (Zhou, 2013). Benefits from peer feedback are found to agree upon alleviation of anxiety, enhancement of self-regulated learning capability and critical thinking competence, increase of peer interactions and active participation, and improvement of overall writing skills (Tsui & Ng, 2000; Cai, 2011; Yu, 2013; Zhou, 2013; Forrer et al., 2015; Wei, 2015; Wang, 2016; Ion et al., 2016; Zhang, 2016). In the meantime, problems are also detected from peer feedback: lack of trust or self-confidence (Xu & Ye, 2014; Wei, 2015; Wang, 2016), impact of face culture that hinders peers giving nega- tive comment (Wei, 2015; Wang, 2016; Xin, 2016), insufficient L2 knowledge (Xu & Ye, 2014; Wang, 2015; Xin, 2016), inadequate ability to implement as- sessment rubrics (Wei, 2015; Xin, 2016), critical thinking deficiency and incom- plete, less-reliable feedback (Bai, 2012; Zhou, 2013; Wang, 2015; Li & Ye, 2016; Wei, 2016; Xin, 2016), all the weaknesses impairing its efficacy. Therefore, peer feedback should be combined with teacher feedback to gain complementary ad- vantages (Yang & Wu, 2011; Yang et al., 2013; Zhou, 2013; Wei, 2015; Xin, 2016; Zhang, 2016). In addition, it is noteworthy that these comparative studies are implemented between peer feedback in pairs or groups and teacher feedback, without distinguishing between peer feedback itself in one study, failing to detect the respective features and problems.

      Automated feedback, a new trend introduced into L2 writing, is designed on

      the basis of large corpora within theoretical principles. Empirical studies find automated feedback, compared with teacher feedback, can provide personalized comments and suggestions, which is more helpful for corrections of linguistics errors (Shi, 2012; Zhou, 2013; Wei, 2015). Comparisons between automated feedback and peer feedback find no significant difference in their contributions to final grades of essays, but automated feedback can trigger more effort to enrich content (Morch et al., 2017). Additionally, students can make progress in linguistic performance, writing competence and self-efficacy with the help of automated feedback (Yang, 2004; Zhou, 2013; Yang & Dai, 2015). Meanwhile researchers also discover weaknesses in automated feedback, for example, it may appear mechanical, inaccurate and repetitive, lessening students’ active adoption (Lai, 2010; Wei, 2015; Morch et al., 2017). Although its efficacy needs more em- pirical studies, there is no denying that automated feedback has gaining mo- mentum in L2 writing practice.

      College English in China is a compulsory course of general education for

      non-English majors in the first and second academic years. Compared with Eng- lish majors, non-English majors do not have the independent writing course, a deficiency in writing skill instruction and practice. Since non-English majors form the majority of college students, their poor writing performance, especially that of lower-intermediate students, requires an urgent treatment. Therefore, to explore an effective feedback package, three feedback conditions are compared: individual


      and group modes in peer feedback and automated feedback. On account of distri- bution features of, students’ perceptions towards and merits-demerits within peer feedback and automated feedback, this paper attempts to seek scaffolding within sociocultural theory in the hope of facilitating Chinese EFL learners’ English writing competence.

  3. Aims and Methodology

    1. Research Questions

      In fact, students’ acceptance and application of feedback in English writing process is influenced by multi-facet dimensions (Lu, 2016), from which the effi- cacy of feedback plays a decisive role. Considering the analysis above, this study aims to address the following questions:

      1. What distribution features are there in peer feedback and automated feed- back?

      2. How do participants perceive peer feedback and automated feedback?


    2. Participants

      In total, participants of this study were engineering sophomores from a local fo- restry university in China, all from one class with mixed majors of bioengineer- ing and environmental engineering. 16 were male and 45 were female with an average age of 20. All the participants had taken the intermediate level of College English Test Band 4, a nationwide English proficiency test, at the first semester of the second academic year and only 20 of them had passed it with a low pass rate of 33%. Therefore, all the participants were lower-intermediate students.

    3. Instruments

      The instruments used in this study include seven essays, Pigai Network, an as- sessment rubric, an open-ended questionnaire and a semi-structural interview. The seven essays were on the same topic, Are We More Connected or More Alone, an argumentative writing of a national composition contest in this May. It was required to be written with no less than 200 words, based on personal un- derstanding of the given material, and taken by all sophomores of the university back then. Pigai Network is an automated evaluation system popularly adopted in Chinese universities and colleges. It is built upon large corpora and the prin- ciples of Noticing Hypothesis and Interaction Theory with established reliability and validity (Hu, 2015; Yang & Dai, 2015). When students post their essays on Pigai Network, it can produce immediate diagnostic feedbacks with marks and comments and show real-time scores, a reflection of qualities of their English writings. An English teacher can access the essays of all the registered students in their class for supervision and necessary intervention. The assessment rubric adopted into this study is a norm in teacher evaluation, aiming to guide students how to evaluate their peers’ writings from content, organization, linguistic per- formance and format. Questionnaires were administered to 61 students and all


      were validly responded, with open-ended questions in relation to students’ per- ceptions towards peer feedback and automated feedback, and the existing merits and demerits within. Semi-structured interviews aimed to better interpret the data collected from peer feedback and questionnaires, 15 participants randomly selected.

    4. Data Collection and Analysis

      This research sampled seven essays titled Are We More Connected or More Alone from Class A. All the samples were of zero modification and poorly eva- luated by Pigai Network, the average score of 74 being less than the holistic, av- erage score of 81. Given their lower-intermediate English level, 61 participants had to be involved into two rounds of feedback in individual and group modes, three or more participants needed in the latter so that the double-evaluated es- says helped to yield adequate, qualified information of peer feedback. In this way, the number of samples was calculated. The English teacher printed them out and deleted all personal information, each paper coded with an Arabic numeral. Before she handed out the samples and the assessment rubrics to all the participants in Class B, instructions were given to participants on how to apply the rubric in essay evaluation, coupled with specific examples and practices on each dimension. Peer feedback in this study was implemented into two modes: individual feedback and group feedback. 14 students were designated to assess the seven essays independently, the rest 47 were required to form 14 groups with 3 or 4 persons a group at their will. Eventually, the seven essays were evaluated twice in either mode. If one marked comment in either mode overlapped with the other in the same place of a single sample, it was treated as a single com- ment. Meanwhile, if marked methods were different, even though aiming at a single spot, they were recorded intact. The individual mode revived how peer feedback in pairs was done online where solo work dominated. In contrast, group mode was a demonstration of how comments were agreed upon through collaborative work. In evaluating phase, both modes were allowed to turn to any source available for help. All the feedback should be finished within 45 minutes and handed in with signatures of reviewers. Afterwards, the questionnaire were administered and retrieved by the English teacher right after they were finished.

      Data from peer feedback, automated feedback and the questionnaire were coded and classified in a word processor. All the results were treated Excel and the statistics were mainly used for the analysis of distribution features of, stu- dents’ perceptions towards, merits and demerits arising from peer feedback and automated feedback.

  4. Results and Discussion

    1. The Distribution Features of Peer Feedback and Automated Feedback

      The distributions of non-corrective feedback and corrective feedback in indi-


      vidual, group and automated modes are displayed in Table 1.

      The descriptive statistics show that automated mode (76.2%) dominates non-corrective feedback, the strong evidence to support that Pigai Network is a very valuable learning tool for EFL students, for its powerful corpora can pro- vide rich linguistic knowledge. Individuals (16.2%) offer more non-corrective feedback than groups (7.6%). The semi-structural interviews reveal that indi- viduals, compared to groups, are influenced more by Chinese culture, avoiding personal offenses and saving the face of the writer. The comments offered in group mode are agreed upon by the whole team, sparing individual members the embarrassment of pointing out their peers’ errors, which agrees with the find- ings of the previous studies (Wei, 2015; Wang, 2016). As for corrective feedback, group mode (46.8%) contributes most, individual mode (33.8%) comes second and automated mode (19.4%) is least helpful. Restricted by participants’ English proficiency, collaborative work in groups helps in detecting errors and giving suggestions than solo work. Members of different English levels become sources of linguistic knowledge; full discussions on problems or uncertainties can dee- pen their understanding and reduce personal responsibility of assessing the es- say. Automated mode is based upon systematic linguistic knowledge distilled from large corpora and its feedback is given sentence by sentence which ends with a dot. But what if the essay is of few grammatical errors and many simple words, or the sentence with a dot is not a legitimate one? In either case, its ability to offer corrective feedback is greatly weakened.

      Considering the way of feedback, group mode ranks top in both direct (53.7%) and indirect (41.7%) feedback, individual mode (39%/35.1%) second and automated mode (7.3%/23.2%) third, the distributions of types are in line with those in holistic corrective feedback. Overall, indirect feedback in individu- al (68), group (81) and automated (45) modes appears more frequently than di- rect feedback with their respective frequencies of 32, 45 and 6. According to in- terviewees, they don’t give direct suggestions on the erroneous points for various


      Table 1. Descriptive statistics of individual mode, group mode and automated mode.


      Non-corrective feedback

      Corrective feedback (percentage)

      (percentage)




      Feedback

      feedback

      34

      32

      68

      21

      81

      89

      (16.20%)

      (39%)

      (35.10%)

      (35%)

      (36.80%)

      (33.80%)

      16

      44

      81

      34

      93

      123

      (7.60%)

      (53.70%)

      (41.70%)

      (56.70%)

      (42.30%)

      (46.80%)

      160

      6

      45

      5

      46

      51

      (76.20%)

      (7.30%)

      (23.20%)

      (8.30%)

      (20.90%)

      (19.40%)

      Total 210

      82

      194

      60

      220

      263

      (100%)

      (100%)

      (100%)

      (100%)

      (100%)

      (100%)

      Direct feedback Indirect feedback Global feedback Local Sum of corrective


      Individual mode


      Group mode


      Automated mode


      justifications: limited knowledge to offer correct solutions, over-confidence in peers to make self-corrections, concerns to intervene in peers’ personal writing styles or just lack of specific requirements from the teacher. Indirect feedback is typically featured in Pigai Network, with its aim to develop students’ self-regulated learning and help internalize linguistic knowledge and rules (Yang, 2004; Yang & Dai, 2015; Wei & Shi, 2016). In terms of information contained in feedback, global feedback unanimously falls behind local feedback in individual (21/81), group (34/93) and automated (5/46) modes, which clearly exposes the short comings in offering feedback and is in line with the previous studies (Zhou, 2013; Wei, 2015; Li & Ye, 2016; Lu, 2016; Wang, 2016; Wei, 2016).

      How dimensions of the assessment rubric are involved in individual, group and automated modes? Table 2 offers a clue.

      Generally, an essay will be assessed from content, organization, linguistic per- formance and format. The former three dimensions are equally important, ac- counting for 90% of its score while the last dimension for 10%. In individual (103), group (90) and automated (169) modes, linguistic performance attracts highest attention, corroborating the findings of relevant studies (Zhou, 2013; Wei, 2015). Specifically, the number of linguistic performance in automated mode (46.6%) is close to the sum of individual (28.5%) and group (24.9%) mod- es, highlighting the outstanding advantage of automated feedback over peer feedback. Therefore, students should be explicitly instructed how to utilize its functions in improving vocabulary, collocation, grammar and syntax in a flexible way, since no errors in vocabulary and grammar is too low a bar for a good writing. Organization in individual (24), group (30) and automated (28) modes is the second important dimension in the judgment of an essay, different from the results found by Zhou (2013) and Wei (2015), indicating subjects of lower English proficiency are less capable of handling content. The performance of group mode (36.6%) is better than that of automated mode (34.1%) and of indi- vidual mode (29.3%), though gap is not wide between the first two modes. When it comes to distributions of content and format in individual, group and auto-


      Table 2. Dimensions of individual mode, group mode and automated mode.



      Content

      Organization

      Linguistic performance

      Format


      9

      24

      103

      2


      (37.50%)

      (29.30%)

      (28.50%)

      (12.40%)


      14

      30

      90

      7


      (58.30%)

      (36.60%)

      (24.90%)

      (43.80%)


      1

      28

      169

      7


      (4.20%)

      (34.10%)

      (46.60%)

      (43.80%)


      24

      82

      362

      16


      (100%)

      (100%)

      (100%)

      (100%)

      Dimensions (percentage)


      Individual mode


      Group mode


      Automated mode


      Total


      mated modes, it is shown that in automated feedback, format (7) is more con- cerned than content while in peer feedback (9/2; 14/7) the reverse is true. The finding echoes the negative evidence against automated evaluation system, which can be fooled and tricked by linguistic performance of an essay with little considering the content of relevance to the topic (Wei, 2015; Morch et al., 2017). In contrast, peers as human can be immune to the tricks, make a judgment on the content and respect the writer’s ideas before giving any suggestion. With re- gards to format, peers are likewise less capable of detecting the mechanical flaws such as insufficient words, imbalance of sentence types, and missing space after punctuation, which can be handled better with technology used in automated feedback. Even so, peers still have an edge in the judgment of essay format by examining whether the essay is divided logically.

      Since focus on linguistic performance and organization is frequent, there is every reason to survey what are the specific aspects covered and how they are distributed in individual, group and automated modes. Table 3 shows vocabu- lary, grammar and syntax in linguistic performance are placed greatest interest in individual mode (42/49), group mode (24/44) and automated mode (118/29). Difference is found in the importance of vocabulary and grammar & syntax in peer feedback and automated feedback. For peers grammar & syntax (49; 44) is more focused than vocabulary (42; 24) and vice versa (118/29) for automated feedback. What should be noted is the missing feedback of punctuation in au- tomated feedback, which reveals its inability to make a judgment on whether a sentence is made logically. However, collocation (77.3%) is a very prominent contribution made by automated feedback, with its corpora as a valid and relia- ble tool to affirm, suggest and correct collocations in essays. Another involved aspect in linguistic performance is spelling mistakes. Surprisingly, holistic peer feedback (37.5%; 37.5%) beats automated feedback (25%) in this aspect. The original data of the three feedback modes reveal that the automated system tar- gets misspelled words alone while peers’ judgment of misspelling goes further,


      Table 3. Aspects of organization and linguistic performance in individual mode, group mode and automated mode.


      Cohesion

      Structure

      Vocabulary

      Collocation

      Grammar & Syntax

      Spelling

      Punctuation

      13

      8

      42

      3

      49

      3

      5

      (21.40%)

      (38.10%)

      (22.80%)

      (13.60%)

      (40.20%)

      (37.50%)

      (31.30%)

      24

      9

      24

      2

      44

      3

      11

      (39.30%)

      (42.90%)

      (13.10%)

      (9.10%)

      (36.10%)

      (37.50%)

      (68.70%)

      24

      4

      118

      17

      29

      2

      0

      (39.30%)

      (19%)

      (64.10%)

      (77.30%)

      (23.80%)

      (25%)

      (0.00%)

      61

      21

      184

      22

      122

      8

      16

      (100%)

      (100%)

      (100%)

      (100%)

      (100%)

      (100%)

      (100%)

      Organization (percentage) Linguistic performance (percentage)


      Individual mode


      Group mode


      Automated mode


      Total


      the textual context being a factor to determine whether the presented word(s) is semantically or grammatically correct.

      Aspects involved in organization lie in cohesion and structure. Concretely, all modes attach greatest importance to cohesion means, an important device to develop an essay in a clear-structured and logical way. Compared with group (39.3%) and automated modes (39.3%), individual mode (21.4%) needs notice this weakness, especially for students of lower English proficiency. Conversely, structure in automated mode (19%) falls far behind it in peer mode (38.1%) and group mode (42.9%). Interviewees explain that in teaching phase English teacher tends to make a detailed analysis of the organization of textbook articles and re- quires students to practice the corresponding aspects with exercises. In evaluat- ing phase, she prefers to focus feedback on essay structure. Naturally, structure falls into students’ attention in assessment of essays.

      Feedback can reflect reviewers’ attitude towards essays. Even though each es- say in this study is double evaluated in two modes of peer feedback, attitude is an independent index to show the stance of its reviewers, therefore, every sample accounts. Table 4 can demonstrate the attitude distributions in individual, group and automated modes.

      In individual mode, affirmation & critique (57.1%) is slight higher than criti- que (42.9%). In group mode, critique (71.4%) appears far more frequently than affirmation & critique (28.6%). Affirmation finds no place in both modes. It is well-known that criticism is of crucial importance for progress-making. Before the evaluating phase, students had been informed the would-be-evaluated essays were not written by classmates, they were put at ease when giving negative comments. If any part of the essay was impressive, affirmation was earned. Since all the samples were more or less flawed, absolute affirmation was unavailable. When group members evaluated the sample, strengths were harder than weak- nesses to be agreed upon and critiques became dominant. In automated mode, affirmation & critique (71.4%) outnumbers affirmation (28.6%), with absolute critique absent. It is easily understandable that every essay has its strengths and weaknesses, related to the lower-intermediate English level of students. The ab- solute affirmation belongs to two samples with few linguistic errors. Such prac- tice may give writers wrong impressions of their writing capability and fail to in- struct them how to polish up their essays.


      Table 4. Attitudes reflected from feedback.


      Attitude reflected from feedback



      Affirmation (percentage)

      Affirmation & critique (percentage)

      Critique (percentage)

      Sum of feed back in each mode (percentage)

      Individual mode

      0

      8 (57.1%)

      6 (42.9%)

      14 (100%)

      Group mode

      0

      4 (28.6%)

      10 (71.4%)

      14 (100%)

      Automated mode

      2 (28.6%)

      5(71.4%)

      0 (0.0%)

      7 (100%)


    2. Students’ Perceptions towards Peer Feedback and Automated Feedback

      All the participants in the study were sophomores with rich experience on how to use Pigai Network, performance on which accounted for 10% of their final grades. Its effects upon writings depend on students’ voluntary adoption of the given comments and suggestions. Although the English teacher, at the very be- ginning of college English course, guided and encouraged students to use it, not every student followed suit for various reasons, lack of teacher supervision hard to be ignored. Comparatively, peer feedback is a less frequent practice for stu- dents, for it requires more complexity and time to implement. For instance, in- structions of assessment rubric, design of assessment process, and teacher su- pervision and teacher intervention, to name a few, are indispensible guarantees. To gather the information on their perceptions towards both means, investiga- tions were mainly made by means of open-ended questionnaires, supplemented with researchers’ real-time observations and analysis on Pigai Network use.

      According to Table 5, most students choose peer feedback (68.9%), some of them (27.8%) refuse it and a few (3.3%) express their uncertainty. As far as au- tomated feedback is concerned, positive attitude belongs to the majority (75.4%), higher than support of peer feedback. By contrast, negative attitude (11.5%) de- clines but uncertain (13.1%) attitude rises. Data show the majority have affirmed the usefulness of peer feedback and automated feedback, confirming the finding in other studies (Zhou, 2013; Wei, 2015; Ion et al., 2016; Lu, 2016; Wang, 2016). Given the status of college English course in China, English teachers are too busy with teaching tasks to evaluate all essays of and offer immediate, complete and personalized feedback to the teaching classes. Under such circumstances, peer feedback and automated feedback are complementary to teacher feedback, for they can provide students with immediate feedback and monitor their writing behavior. In the evaluating phase, most of participants were observed to read, discuss or consult dictionaries before writing down the feedback. Data from Pig- ai Network further confirmed the majority (79%) had made modifications with an average frequency of 4.5 times. In the meantime, weakness from peer feed- back and automated feedback, low ability to apply assessment rubric, lack of teacher supervision, and faulty assessment process and insufficient teacher in- tervention beget students’ negative attitude towards them. For instance, feedback given to some samples did not contain detailed comments or suggestions. The number that the teacher intervened through Pigai Network is much smaller than



      Uncertain

      Negative

      Total

      (Percentage)

      (Percentage)

      (Percentage)

      Perceptions to peer feedback

      42 (68.90%)

      2 (3.30%)

      17 (27.80%)

      61 (100%)

      Perceptions to automated feedback

      46 (75.40%)

      8 (13.10%)

      7 (11.50%)

      61 (100%)

      Table 5. Perceptions towards peer feedback and automated feedback.


      Positive (Percentage)


      that without teacher’s interventions (33%/67%). Less than 10 students turned to the teacher for help during the whole process, even though the teacher walked around and stopped to listen from time to time. The modification frequency higher than the average frequency (29.5%) is not in dominant position.

    3. Merits and Demerits Arising from Peer Feedback and Automated Feedback

      Based on the analysis of open-ended questionnaires, four main merits are sum- marized on peer feedback. Firstly, students consider it as an opportunity to learn from peers on grammar, syntax, vocabulary and learning strategy, and to apply the learned knowledge into practice. Secondly, errors made by peers are warn- ings against the same repetitions in their own essays. Then, by changing the role from a writer to a reader and a reviewer, personal ideas on a topic can be better communicated, views broadened, writers’ sense of audience enhanced and un- derstanding of the theme deepened, a good way to develop critical thinking. Fi- nally, during the process, their reading, writing and communicating skills are advanced.

      Similarly, three significant advantages are harvested from automated feed- back. Firstly, it can locate linguistic, grammatical and syntactic errors accurately with few mistakes, whose indirect suggestions force students to do some re- search before self-corrections. Such practice can strengthen individual under- standing of the language points and boost self-regulated learning. Secondly, non-corrective feedback on vocabulary helps students enlarge vocabulary on advanced words through synonyms, antonyms and differentiation of words, a way that helps build a mental lexicon network, connecting prior vocabulary with newly-learned vocabulary. Thirdly, following suggestions can lead to improved linguistic performance and increase the essay scores, gradually the writing skill improved in certain aspects.

      With merits above, demerits should be given attention as well. According to participants’ reports, the top concern on peer feedback is low English proficiency of peers, which may cast a negative impact on validity and reliability of feedback, hinder them from offering complete and constructive corrective feedback, and restrict the range of feedback mainly into vocabulary, grammar and syntax. Evi- dence can be found from data of dimensions from individual, group and auto- mated modes in Table 2. Secondly, diverse opinions on an aspect are hard to ar- rive at agreement, which may cause rejection of feedback without careful con- sideration. Thirdly, lack of supervision, effective assessment process and neces- sary teacher intervention may invite students of less interest or ability to give flippant comments on peers’ essays. A small number of students further express their worries over loss of characteristic styles, worsening interpersonal relation- ship and plagiarism. Differently, the biggest problem from automated feedback is its rigid, mechanical, inconsistent suggestions, thus less-reliable scores. Similar concerns go to lack of supervision, effective assessment process and teacher in-


      tervention, reducing its effectiveness in essay improvement. For less-skilled stu- dents, too much indirect corrective feedback on grammar and syntax inflicts unbearable burden in solo work. Lastly, it has very limited capacity to offer feedback on content and organization evidenced in Table 2 and Table 3.

    4. Feedback Scaffolding within the Sociocultural Theory

      Based on the findings above, feedback from different sources should be com- bined to provide scaffolding to facilitate writing skills of lower-intermediate students within sociocultural theory.

      As subject, students are bound by their low English proficiency. To produce valid and reliable feedback for peers, resources from English learners’ dictiona- ries are of great help in providing scaffolding in content, structure and linguistic performance (Wei, 2016). In the process, students’ linguistic knowledge can be increased, accordingly, their initiatives and subjective self-consciousness streng- thened.

      As object, essays are the premise and result of interactions. Students are mo- tivated to interact with their surroundings to improve their writing skills. Such a goal can be achieved by emphasizing both corrective and non-corrective feed- back. Thus, feedback should go beyond error corrections, rather as a way to ac- cumulate linguistic knowledge.

      As rules, rubric and process mechanism for assessment should be explicitly informed and strictly implemented. Rubric instructions help students make ef- fective feedback for peers while process mechanism can reduce students’ careless responses to feedback.

      As components of learning community, teachers and peers can offer scaffold- ing by providing feedback through interactions. Teachers can interact with stu- dents in the following ways. Firstly, teachers should be diverted from the role of sole feedback source to a designer, supporter, supervisor and reviewer. They should be responsible for designing feedback activities and assessment process to raise students’ interest and willingness of participation, guarantee friendly at- mosphere, give emotional and material support, supervise students’ behavior and response to feedback and intervene when necessary. Compared with stu- dents, teachers are more experienced and professional. To make up for defects of automated feedback in making comments on content and organization, instruc- tions and exercises should be provided to enhance students’ awareness and abil- ity in this regard. Given insufficient global feedback in peer feedback and auto- mated feedback, explicit instructions should be offered with demonstrations through analysis of textbook articles and good essays written by students and corresponding exercises as a test of learned knowledge. In this way, students’ sensitivity to global issues can be strengthened, active application in peer feed- back increased. Finally, when the teacher assigns essays for peer feedback in pairs, poor written essays had better be designated to high performers and well-written essays to poor performers. Thus, teachers can create opportunities


      for poor performers to learn from their more capable peers and for high perfor- mers to practice their learned knowledge. If feedback is in group mode, teachers should guarantee the group is better to consist of students of different English levels so that mutual benefits can be gained in collaborative work.

      Meanwhile students’ interactions with peers include frequent communication by negotiations and discussions. Misunderstandings, doubts, disagreements, and fear of loss of characteristic styles can be dissolved when interpersonal commu- nication works well and fully. Students should realize disputes between peers should not be a big concern, for disagreement itself helps active and critical thinking and respect for disagreement is also a way to broaden personal views. Considering the poor quality of an essay, students can increase direct feedback and provide detailed explanation or suggestions.

      As a mediation tool, Pigai Network can provide scaffolding by interactions with students. Firstly, corrective feedback and non-corrective feedback help stu- dents realize its shortcomings and strengths. Particularly, information on voca- bulary, collocation, differentiation of synonyms, natural English expressions, etc. is conducive to language knowledge accumulation. Students should value and utilize them in an active manner. Its prominent feature in indirect feedback can promote self-corrections and self-reflection upon errors. Notices of errors can test students’ learned knowledge, search for answers enhancing their initiatives in English learning. In view of inaccurate and mechanical feedback offered in Pigai Network, students should make a judgment before taking them. If they don’t agree with the comment, they had better do some research first. Once con- firming the wrong feedback, they can report the errors online. In this way, stu- dents are actively interactive with the Network, instead of receiving everything passively from it.

      In division of labor, students are changed from writers to readers, reviewers and learners in feedback. They should not pay attention to errors alone, but also should recognize sparking points in essays, for critique is helpful in correcting errors and affirmation in increasing knowledge and motivating peers’ participa- tion (Wei, 2015; Luo, 2016). Teachers take roles of instructors, participants, faci- litators and assistants, creating various scenarios to smooth feedback for stu- dents. The changing roles turn teachers and students to scaffolding providers.

  5. Conclusion

The research analyzes distribution features of peer feedback and automated feedback from types, dimensions, attitudes. It is found that as for non-corrective feedback automated mode dominates while for corrective feedback group mode has the lion’s share. In terms of direct and indirect feedback, group mode is in the lead ahead of individual and automated modes, in line with that in holistic corrective feedback. Specifically, indirect feedback appears more than direct feedback in peer feedback and automated feedback. In respect to global feedback and local feedback, the imbalance exists in individual, group and automated


modes. Dimension distributions in peer feedback and automated feedback are found to centre on linguistic performance, followed by organization, content and format. Further investigations into linguistic performance find that vocabu- lary, grammar and syntax draw greatest attention in peer feedback and auto- mated feedback. Collocation, spelling and punctuation are minor aspects, each behaving differently in individual, group and automated modes. In organization, cohesion means are found to work poor in individual mode while automated feedback enjoys the same status in structure. Attitude distributions perform dif- ferently in peer feedback and automated feedback. The former leaves no space for absolute affirmation whereas the latter expels absolute critique. Even in peer feedback, difference exists in the proportion of affirmation & critique to critique between individual and group modes.

The results of students’ perceptions towards peer feedback and automated feedback show the majority hold positive attitude, with the minority having neg- ative or uncertain attitude. Varying attitudes from student to student can be ac- counted for by the found merits and demerits within. Peer feedback is valued as learning opportunities, warnings against repeated errors and means of mul- ti-role interactions and skill improvement. Similarly, automated feedback is ap- preciated for its functions to accurately locate errors, offer abundant lexical knowledge and suggest constructive corrective feedback. Meanwhile, demerits should not be underrated in peer feedback and automated feedback. Main find- ings include limited English proficiency of peers, careless response to the con- flicting opinions, lack of supervision and effective assessment process, rigid and inconsistent feedback, preference to indirect corrective feedback and the insuffi- cient ability to comment on organization.

On the basis of the findings above, key elements from sociocultural theory are explored to provide feedback scaffolding to improve students’ writing skills. All in all, teacher, peer and automated feedback should be combined to set up mul- ti-dimensional scaffolding by fully utilizing their respective merits. Appropriate and flexible procedures need to be further explored based on action research. Only in this way can feedback scaffolding work best to promote students’ writ- ing competence.

Acknowledgements

This research is supported by Education Office of Yu Nan province in Scientific Research Foundation (No. 2015Y312)

References

Andrade, H. (2008). Self-Assessment through Rubrics. Educational Leadership, 4, 60-63. Aryadoust, V., & Riazi, M. (2017). Role of Assessment in Second Language Writing Re-

search and Pedagogy. Educational Psychology, 37, 1-7. https://doi.org/10.1080/01443410.2016.1227089

Bai, L. R. (2012). Validation of Peer Feedback Pattern Inventory for College English Writing. Modern Foreign Languages, 35, 184-192.


Black, P., & William, D. (2009). Developing the Theory of Formative Assessment. Educa- tional Assessment, Evaluation and Accountability, 21, 5-31. https://doi.org/10.1007/s11092-008-9068-5

Cai, J. G. (2011). A Comparative Study of Online Peer Feedback and Teacher Feedback for Chinese College Students’ English Writing. Foreign Language World, 2, 65-72.

Chandler, J. (2003). The Efficacy of Various Kinds of Error Feedback for Improvement in the Accuracy and Fluency of L2 Student Writing. Journal of Second Language Writing, 12, 267-296.

Chang, C. Y. (2015). Teacher Modeling on EFL Reviewers’ Audience Awareness Feedback and Affectivity in L2 Peer Review. Assisting Writing, 25, 1-20.

Chapelle, C. A., Cotos, E., & Lee, J. Y. (2015). Validity Arguments for Diagnostic Assess- ment Using Automated Writing Evaluation. Language Testing, 32, 385-405. https://doi.org/10.1177/0265532214565386

Cho, K., & Schunn, C. (2010). Developing Writing Skills through Students Giving In- structional Explanations. In M. K. Stein, & L. Kucan (Eds.), Instructional Explanations in the Disciplines (pp.207-221). New York, NY: Springer.

https://doi.org/10.1007/978-1-4419-0594-9_13

Coyle, Y., & Roca de Larios, J. (2014). Exploring the Role Played by Error Correction and Models on Children’s Reported Noticing and Output Production in a L2 Writing Task. Studies in Second Language Acquisition, 3, 451-485. https://doi.org/10.1017/S0272263113000612

Ellis, R., Sheen, Y., Murakami, M., & Takashima, H. (2008). The Effects of Focused and Unfocused Written Corrective Feedback in an English as a Foreign Language Context. System, 36, 353-371.

Engeström, Y. (1987). Learning by Expanding: An Activity Theoretical Approach to De- velopmental Research. Helsinki: Orienta-Konsultit Oy.

Engeström, Y. (2001). Expansive Learning at Work: Toward an Activity Theoretical Re- conceptualization. Journal of Education and Work, 14, 133-156. https://doi.org/10.1080/13639080020028747

Ferris, D. R. (2003). Response to Student Writing: Implications for Second-Language Students. Mahwah: Lawrence Elbaum Associates, Inc.

Forrer, D. A., Wyant, N. A., & Smith, M. G. (2015). Improving Writing through the Peer-to-Peer Evaluation Process. Journal of Business & Economics Research, 3, 137-143. https://doi.org/10.19030/jber.v13i3.9285

Hu, X. W. (2015). The Effect of Self-Modification on College English Writing Based on Automated Essay Evaluation. Computer-Assisted Foreign Language Education, 3, 45-49.

Ion, G., Barrera-Corominas, A., & Tomàs-Folch, M. (2016). Written Peer-Feedback to Enhance Students’ Current and Future Learning. International Journal of Educational Technology in Higher Education, 13, 1-11.

https://doi.org/10.1186/s41239-016-0017-y

Keh, C. L. (1990). Feedback in the Writing Process: A Model and Methods for Imple- mentation. ELT Journal, 4, 294-394. https://doi.org/10.1093/elt/44.4.294

Lai, Y. (2010). Which Do Students Prefer to Evaluate Their Essays: Peers or Computer Program. British Journal of Educational Technology, 41, 432-454. https://doi.org/10.1111/j.1467-8535.2009.00959.x

Lantolf, J. P., & Thorne, S. L. (2006). Sociocultural Theory and He Genesis of Second Language Development. Oxford: Oxford University Press.


Leggette, H. R., McKim, B. R., & Dunsford, D. (2013). A Case Study of Using Electronic Self-Assessment Rubrics in a Core Curriculum Writing Course. North America Col- leges and Teachers of Agriculture Journal, 6, 2-10.

Li, L., & Ye, J. M. (2016). A Study of Language Expression Feedback Difference in College English Writing. Shandong Foreign Language Teaching, 37, 58-64.

Liao, H. C., & Yang, C. C. (2012). Teaching Practice and Cultural Difference of an English as Foreign Language Classroom in Taiwan. English Language Teaching, 11, 151-160. https://doi.org/10.5539/elt.v5n11p151

Liu, F. (2016). A Study on the Effect of Quantitative Teacher Intervention towards Col- lege English Writing. Foreign Language Research, 188, 110-115.

Lu, L. (2016). A Case Study of L2 Writing Process Based on an Automated Essay Evalua- tion Tool. Foreign Language World, 2, 88-96.

Luo, S. (2016). Implementing Scaffolded Peer Evaluation in MOOC: A Case with EFL Writing Tasks. Computer-Assisted Foreign Language Education, 172, 42-47.

Morch, A. I., Engeness, I., Cheng, V. C., Cheung, W. K., & Wong, K. C. (2017). Essay- Critic: Writing to Learn with a Knowledge-Based Design Critiquing System. Educa- tional Technology & Society, 20, 213-223.

O’Mahony, B., Verezub, E., Dalrymple, J., & Bertone, S. (2013). An Evaluation of Re- search Students’ Writing Support Intervention. Journal of International Education in Business, 6, 22-34. https://doi.org/10.1108/18363261311314935

Plank, S. B., & Condliffe, B. (2011). Pressures of the Season: A Descriptive Look at Class- room Quality in Second and Third Grade Classrooms. Baltimore, MD: Baltimore Edu- cation Research Consortium.

Raoofi, S., Binandeh, M., & Rahmani, S. (2017). An Investigation into Writing Strategies and Writing Proficiency of University Students. Journal of Language Teaching and Re- search, 8, 191-198. https://doi.org/10.17507/jltr.0801.24

Rollinson, P. (2005). Using Peer Feedback in the ESL Writing Class. ELT Journal, 59,

23-30. https://doi.org/10.1093/elt/cci003

Sarie, R. F. (2017). Exploring the Possibility of Using both Direct and Indirect Corrective Feedback as Strategy to Improve Students’ Writing Skill in Indonesian Secondary School. International Journal of Arts and Sciences, 9, 597-604.

Shi, X. L. (2012). Research on the Application of the Automatic Writing System in Col- lege English Writing Teaching: A Case Study with Pigai Network. Modern Educational Technology, 10, 67-71.

Tsui, A. B., & Ng, M. (2000). Does Secondary L2 Writers Benefit from Peer Comments?

Journal of Second Language Writing, 2, 147-170.

Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press.

Wertsch, J. V. (1985). Vygotsky and the Social Formation of Mind. Cambridge, MA: Harvard University Press.

Wang, W. (2016). Peer Feedback in Chinese English Writing Class: Using Action Re- search to Promote Students’ English Writing. Journal of Language Teaching and Re- search, 7, 958-966. https://doi.org/10.17507/jltr.0705.17

Wang, X. L. (2015). A Study on Types and Characteristics of Students’ Written Feedback by Peer-Revision in English Writing. Journal of Beijing City University, 3, 55-59, 65.

Wang, Y., & Liu, Z. Q. (2012). The Effects of Teacher Feedback on Accuracy, Fluency, Complexity and Overall Quality of English Writing. Foreign Language Education, 6,


49-53.

Wei, M. (2015). An Empirical Study into Effects of Feedback on College English Writing.

Journal of Tianjin Foreign Studies University, 22, 43-50.

Wei, X. B., & Shi, Q. B. (2016). Approaches, Problems and Prospects of L2 Written Feed- back Research at Home and Abroad. Foreign Language World, 5, 28-36.

Wei, X. Q. (2016). An Investigation into Scaffolding Functions of Learners’ Dictionaries in L2 Writing Teaching. Journal of Lexicography, 3, 1-10.

Xin, S. (2016). Peer Assessment Accuracy and Self-Revision Differences of English Writ- ing. Journal of Guangdong University of Foreign Studies, 7, 64-70.

Xu, J. F., & Ye, M. S. (2014). A Study on Peer Interaction in ESL/ EFL Classroom. Con- temporary Foreign Languages Studies, 10, 31-36.

Xu, Y. T., & Liu, J. (2010). Research on L2 Writing Feedback Based on Anonymous Writ- ten Feedback. Foreign Language Teaching, 3, 44-49.

Yang, L. J., Yang, M. J., & Zhang, Y. (2013). A Comparative Study of Teacher Feedback, Peer Feedback and Teacher-Peer Combined Feedback in EFL Writing in China. For- eign Language Education, 3, 63-67.

Yang, M. (2006). A Comparative Study of Teacher Feedback and Peer Feedback in Chi- nese EFL Writing Class. Modern Foreign Languages, 3, 293-301.

Yang, N. D. (2004). Using MyAcess in EFL Writing. In Proceedings of the 2004 Interna- tional Conference and Workshops on TEFL & Applied Linguistics (pp. 550-564). Tai- pei: The Crane Publishing Co., Ltd.

Yang, X. Q., & Dai, Y. C. (2015). Practice Research on Self-Regulated Writing Model of College English Teaching Based on Pigai Network. Computer-Assisted Foreign Lan- guage Education, 2, 17-23.

Yang, Y. F., & Wu, S, P. (2011). A Collective Case Study of Online Interaction Patterns in Text Revisions. Educational Technology & Society, 14, 1-15.

Yu, S. L. (2013). Teacher Feedback and Peer Feedback: Differences and Integration from the Perspective of Social Cultural Activity Theory. Modern Foreign Languages, 2, 70-76.

Zhang, W. L. (2016). A Peer Response Writing Activity for College Level Japanese Classes. Japanese Learning and Research, 182, 86-91.

Zhou, Y. S. (2013). A Comparative Study of Teacher Feedback, Peer Feedback and Online Feedback in English Writing. Foreign Language World, 3, 87-96.

Zhu, Q. J. (2010). Feedback Mechanisms and Teaching of Writing: Implications of For- eign and Domestic Research on College English Writing Teaching. Journal of Chang- chun University of Science and Technology, 3, 160-162.

Submit or recommend next manuscript to SCIRP and we will provide best service for you:

Accepting pre-submission inquiries through Email, Facebook, LinkedIn, Twitter, etc. A wide selection of journals (inclusive of 9 subjects, more than 200 journals) Providing 24-hour high-quality service

User-friendly online submission system Fair and swift peer-review system

Efficient typesetting and proofreading procedure

Display of the result of downloads and visits, as well as the number of cited articles Maximum dissemination of your research work

Submit your manuscript at: http://papersubmission.scirp.org/ Or contact ojml@scirp.org